Papers with imitation learning
Unveiling Imitation Learning: Exploring the impact of Data Falsity to Large Language Model (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on improving open-source language models through imitation learning. |
| Approach: | They propose to use false pairs to control the factuality ratio of a dataset to study noise. |
| Outcome: | The proposed model is based on a Falsity-Controllable dataset with false pairs and false pairs to control the factuality ratio. |
Improving Retrospective Language Agents via Joint Policy Gradient Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have sparked interest in creating autonomous agents. |
| Approach: | They propose a framework that jointly optimizes both task-planning and self-reflective evolution capabilities in language agents. |
| Outcome: | The proposed framework improves task planning and self-reflective evolution capabilities in language agents. |
Seeded self-play for language learning (D19-64)
Copied to clipboard
| Challenge: | Current methods for learning human language are too data inefficient to learn it in this way. |
| Approach: | They propose to train a meta-learning agent in simulation to interact with populations of pre-trained agents, each with their own distinct communication protocol. |
| Outcome: | The proposed algorithm minimizes the number of on-policy interactions while learning human language while minimizing the number on-political interactions. |
EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints (2021.tacl-1)
Copied to clipboard
| Challenge: | Empirically, EDITOR uses soft lexical constraints more effectively than the Levenshtein Transformer while speeding up decoding dramatically compared to constrained beam search. |
| Approach: | They propose an Edit-Based TransfOrmer with Repositioning that integrates lexical preferences into output sequences by iterative editing hypotheses. |
| Outcome: | The proposed model uses soft lexical constraints more effectively than the Levenshtein Transformer while speeding up decoding dramatically compared to constrained beam search. |
LEAP & LEAN: Look-ahead Planning and Agile Navigation for LLM Agents (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing approaches to train large-scale models with extensive datasets are limited by their inadequate planning capabilities compared to humans. |
| Approach: | They propose a paradigm that enhances the performance of Large Language Models (LLMs) they use look-ahead planning to refine action selection and LEAN to streamline navigation through agile prompt construction. |
| Outcome: | The proposed framework outperforms agents trained via imitation learning, reinforcement learning, and reasoning-based approaches without any fine-tuning. |
From Coarse to Fine: Self-Adaptive Hierarchical Planning for LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing plans for large language model-based agents are limited by their granularity and lack flexibility. |
| Approach: | They propose a self-adaptive hierarchical planning mechanism that mimics human planning strategies and generates self-adapted hierarchic plans tailored to the varying difficulty levels of different tasks. |
| Outcome: | The proposed method significantly improves task execution success rates while mitigating overthinking at the planning level, providing a flexible and efficient solution for multi-step complex decision-making tasks. |
Understanding Game-Playing Agents with Natural Language Annotations (2022.acl-short)
Copied to clipboard
| Challenge: | a new dataset contains 10K human-annotated Go games. |
| Approach: | They propose to use natural language annotations to predict game terms from Go models . they use linear probing to predict domain-specific terms from the model's intermediate state representations . |
| Outcome: | The proposed model can predict domain-specific terms from human-annotated Go games . the proposed model is based on a dataset containing 10K human-rannotated games based in the game . |
LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models have shown significant potential for robotics tasks, but a gap remains in personalization of LLMs to household preferences. |
| Approach: | They propose a framework to personalize LLM planners for household robotics . they use imitation learning and reinforced self-training to personalise the planner . |
| Outcome: | The proposed framework performs iterative planning in multi-room, partially-observable household environments, utilizing a scene graph built dynamically from local observations. |
Embodied Executable Policy Learning with Language-based Scene Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing Large Language models with text inputs lack the capability to evolve with non-expert interactions with environments. |
| Approach: | They propose a novel learning paradigm that generates robots’ executable actions in the form of text, derived solely from visual observations. |
| Outcome: | The proposed learning paradigm surpasses baselines and can adapt to the target tasks effectively. |
Towards Example-Based NMT with Multi-Levenshtein Transformers (2023.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-augmented machine translation (RAMT) is attracting growing attention . it is assumed to implement some form of domain adaptation . |
| Approach: | They propose a retrieval-augmented version of the Levenshtein Transformer to make it more transparent . they propose to perform training and inference in this model, based on multi-way alignment algorithms and imitation learning. |
| Outcome: | The proposed architecture improves translation performance and improves consistency of translations compared to previous models. |
Text Editing as Imitation Game (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text editing is an important domain of processing tasks to edit the text in a localized fashion, such as text simplification. |
| Approach: | They propose a nonautoregressive decoder for state-to-action demonstrations that parallels the decoding while retaining the dependencies between tokens. |
| Outcome: | The proposed model outperforms the autoregressive baselines on a suite of Arithmetic Equation benchmarks in terms of performance, efficiency, and robustness. |
VISITRON: Visual Semantics-Aligned Interactively Trained Object-Navigator (2022.findings-acl)
Copied to clipboard
Ayush Shrivastava, Karthik Gopalakrishnan, Yang Liu, Robinson Piramuthu, Gokhan Tur, Devi Parikh, Dilek Hakkani-Tur
| Challenge: | Interactive robots navigating photo-realistic environments need to be trained to handle dynamic nature of dialogue and vision-and-language navigation (VLN). |
| Approach: | They propose a Transformer-based multi-modal navigator that is better suited to the interactive regime inherent to Cooperative Vision-and-Dialog Navigation (CVDN). |
| Outcome: | The proposed model is trained to identify and associate object-level concepts and semantics between the environment and dialogue history and identify when to interact vs. navigate via imitation learning of a binary classification head. |
Learning How to Actively Learn: A Deep Imitation Learning Approach (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that heuristic-based active learning methods are limited when the data distribution of the underlying learning problems vary. |
| Approach: | They propose a method that learns an AL "policy" using "imitation learning" they use an efficient "algorithmic expert" which provides the policy learner with good actions in the encountered AL situations. |
| Outcome: | The proposed method is more effective than previous methods on two tasks . labeled data is rare while unlabelled data is abundant . |
Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems (N18-1)
Copied to clipboard
| Challenge: | Existing methods for learning task-oriented dialogues include applying reinforcement learning with user feedback on supervised pre-training models. |
| Approach: | They propose a hybrid imitation and reinforcement learning method that integrates user feedback and reinforcement training to improve the agent's performance. |
| Outcome: | The proposed method can learn from the mistake it makes via imitation learning from user teaching and feedback. |
BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps (2020.acl-main)
Copied to clipboard
| Challenge: | Existing state-of-the-art VLN agents do not generalize well for long navigation tasks. |
| Approach: | They propose a VLN agent that is learned to navigate by decomposing long instructions into shorter ones and completing them sequentially. |
| Outcome: | The proposed agent can follow long instructions better than existing ones, but it does not generalize well. |
Learning Coupled Policies for Simultaneous Machine Translation using Imitation Learning (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to learn simultaneous translation model with coupled programmer-interpreter policies are suboptimal as they fix the agent's policy to focus learning the NMT model or learn adaptive agent policies while the NRT model is fixed. |
| Approach: | They propose an algorithmic oracle to produce oracular READ/WRITE actions for training bilingual sentence-pairs using the notion of word alignments. |
| Outcome: | The proposed method outperforms baselines in terms of translation quality quality while keeping the delay low. |
WebWISE: Unlocking Web Interface Control for LLMs via Sequential Exploration (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prior work to control software has used reinforcement learning (RL), requiring many demonstrations and trials to learn simple interaction tasks. |
| Approach: | They propose a Large Language Model to automatically perform web software tasks using click, scroll, and text in- put operations using filtered Document Object Models as observations. |
| Outcome: | The proposed method performs better on the MiniWob++ benchmark with only one in-context example. |
Learning to Generalize for Sequential Decision Making (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances show strong evidence of generalization in spatiotemporal modalities such as robotic manipulation. |
| Approach: | They propose a method for converting a reinforcement learning model into a natural language understanding model by a teacher-student imitation learning method. |
| Outcome: | The proposed model outperforms teacher performance on held-out decision problems by 7% and 24% on out-of-domain problems. |
Imitation Learning for Neural Morphological String Transduction (D18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that neural transition-based models can be used for morphological tasks such as inflection generation and lemmatization without a character aligner or warm start. |
| Approach: | They propose to use imitation learning to train a neural transition-based string transducer for morphological tasks such as inflection generation and lemmatization. |
| Outcome: | The proposed model eliminates the need for a character aligner or warm start and achieves state-of-the-art performance on several datasets. |
Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments on challenging mathematical reasoning benchmarks demonstrate that these human-inspired strategies synergistically and significantly enhance performance. |
| Approach: | They propose to use Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation to improve model performance. |
| Outcome: | Extensive experiments on mathematical reasoning benchmarks show that the proposed strategies synergistically and significantly improve performance over the baseline model. |
Tackling the Low-resource Challenge for Canonical Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | morphological segmentation is a task of dividing words into their constituting morphemes . we compare two new approaches for the task when training data is limited . |
| Approach: | They propose to use an LSTM pointer-generator and a sequence-to-sequence model to perform canonical segmentation when training data is limited. |
| Outcome: | The proposed models outperform existing models on German, English, and Indonesian in low-resource scenarios by 11.4% accuracy. |
DEMO: Reframing Dialogue Interaction with Fine-grained Element Modeling (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) enabled dialogue systems are one of the central modes in human-machine interaction. |
| Approach: | They propose a benchmark task for dialogue element MOdeling and Element Awareness and a new benchmark for dialogue agent interaction that allows the agent to model dialogue elements via imitation learning. |
| Outcome: | The proposed agent performs well in both dialogue element modeling and out-of-domain tasks. |
An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation (D19-1)
Copied to clipboard
| Challenge: | Existing methods to generate paraphrases are not trivial and often fail in practice. |
| Approach: | They propose to use imitation learning to boost the performance of generating paraphrases by using a pointer-generator model. |
| Outcome: | The proposed model outperforms the state-of-the-art methods on the benchmark datasets. |
Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large reasoning models have exhibited strong performance on complex reasoning tasks, but current test-time scaling methods rely on redundant sampling and ignore historical experience utilization. |
| Approach: | They propose a test-time scaling framework that coordinates three collaborative LRMs to iteratively explore and refine solutions guided by historical attempts. |
| Outcome: | The proposed framework surpasses strong baselines on three mathematical reasoning benchmarks, including AIME-24, AIME-25, and OlymMATH. |
Don’t Copy the Teacher: Data and Model Challenges in Embodied Dialogue (2022.emnlp-main)
Copied to clipboard
| Challenge: | Embodied dialogue instruction following requires an agent to complete a complex sequence of tasks from a natural language exchange. |
| Approach: | They argue that imitation learning and low-level metrics are misleading . they compare existing models with IL and argue evaluation should focus on higher-level semantic goals . |
| Outcome: | The proposed model evaluations are based on three models and compare them with benchmarks . they show that existing models fail to ground query utterances, which are essential for task completion . |
Learning from Mistakes via Cooperative Study Assistant for Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated their potential to refine their generation based on their own feedback, but the feedback from LLM itself is often inaccurate, thereby limiting its benefits. |
| Approach: | They propose a framework with an auxiliary agent to assist the main LLM in learning from mistakes through interactive cooperation. |
| Outcome: | The proposed framework can significantly boost large language models by an accuracy margin of up to 6.6 on BBH and 12.6 on BBQ. |
Astra: Efficient Transformer Architecture and Contrastive Dynamics Learning for Embodied Instruction Following (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing vision-language-action models rely on causal attention for processing sequences composed of interleaved segments from different modalities. |
| Approach: | They propose a Transformer architecture featuring trajectory attention and learnable action queries that efficiently process segmented multimodal trajectories and predict actions for imitation learning. |
| Outcome: | The proposed architecture performs better on three large-scale robot manipulation benchmarks than previous models. |
How to Leverage Demonstration Data in Alignment for Large Language Model? A Self-Imitation Learning Perspective (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models are not suitable for task-dependent tasks. |
| Approach: | They propose a generalized self-imitation learning framework which aligns large language models with offline demonstration data. |
| Outcome: | The proposed framework outperforms baselines in many challenging benchmarks . it is available on github.com/tengxiao1/GSIL . |
On the Correspondence between Compositionality and Imitation in Emergent Neural Communication (2023.findings-acl)
Copied to clipboard
| Challenge: | a study examining compositionality and imitation learning in a Lewis game demonstrates that it is difficult to imitate compositional languages. |
| Approach: | They explore the link between compositionality and imitation in a Lewis game . they show that the learning algorithm used to imitate is crucial . |
| Outcome: | The proposed model improves compositionality and imitation in a Lewis game . the study shows that compositional languages are easier to imitate . |
JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse (2025.findings-acl)
Copied to clipboard
| Challenge: | Visual Language Action models have shown promise in decision-making tasks, but have been neglected in previous work . |
| Approach: | They propose a new paradigm for visual language action models that enhances the foundation model prior to action-specific tuning by first post-training it on a curated set of visual and linguistic tasks using self-supervised learning. |
| Outcome: | The proposed model outperforms the best agent baseline on a diverse set of atomic tasks and surpasses imitation learning-based policies in Minecraft. |
FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current approaches to embodied AI tend to learn policies from expert demonstrations, but without a mechanism to evaluate the quality of demonstrated actions, they are limited to learning from optimal behaviour or risk replicating errors and inefficiencies. |
| Approach: | They propose to embed language feedback into a Transformer-based policy and optionally complement the traditional next action prediction objective with auxiliary self-supervised learning objectives for feedback prediction. |
| Outcome: | The proposed method improves agents’ compositional generalisation abilities and robustness on a range of embodied Vision-and-Language tasks in a custom babyAI-XGen environment. |
OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization (2025.acl-long)
Copied to clipboard
Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Hongming Zhang, Tianqing Fang, Zhenzhong Lan, Dong Yu
| Challenge: | Existing studies focus on building text-only agents in synthetic environments where the reward signals are clearly defined. |
| Approach: | They propose a multimodal web agent that can autonomously conduct real-world exploration and improve itself after each iteration. |
| Outcome: | The proposed agent improves itself after each iteration, demonstrating strong performance across multiple test sets. |
EMTIR-GRPO: Efficient Multi-Tool Augmented Large Language Models via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Tool-integrated reasoning (TIR) enables large language models to invoke external tools for tasks beyond their internal capacity but often suffers from tool overuse. |
| Approach: | They propose an algorithm that uses a composite reward to model tool costs and tool efficiency. |
| Outcome: | The proposed algorithm models heterogeneous tool costs and encourages more cost-effective tool-use strategies. |
Playpen: An Environment for Exploring Learning From Dialogue Game Feedback (2025.emnlp-main)
Copied to clipboard
Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Momentè, Luca Franceschetti, Philipp Sadler, Sherzod Hakimov, Alberto Testoni, Raffaella Bernardi, Raquel Fernández, Alexander Koller, Oliver Lemon, David Schlangen, Mario Giulianelli, Alessandro Suglia
| Challenge: | In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed activities driven predominantly by verbal actions—can also serve as a source of feedback signals for learning. |
| Approach: | They introduce Playpen, an environment for off- and online learning through Dialogue Game self-play, and investigate a representative set of post-training methods: supervised fine-tuning, direct alignment and reinforcement learning with Group Relative Policy Optimization. |
| Outcome: | The proposed model improves performance on unseen instances, but negatively impacts other skills, while interactive learning shows balanced improvements without loss of skills. |
VIDA: A Visual Intent-driven Design Assistant for Proactive Multimodal Clarification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing vision-language models fail to provide accurate and complete answers to user requests . a new strategy-aware design assistant is developed to help designers create proactive, visually grounded, and strategically prioritized clarification questions. |
| Approach: | They propose a visual intent-driven design assistant to generate proactive, visually grounded, and strategically prioritized clarification questions. |
| Outcome: | The proposed assistant improves the strategic alignment score by 20.59% over baselines and restores visual grounding capabilities lost during fine-tuning. |
Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks (2026.acl-long)
Copied to clipboard
Wenqi Zhang, Mengna Wang, Gangao Liu, Huixin Xu, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Jiajun Liu, Weiming Lu, Peng Li, Yueting Zhuang
| Challenge: | Recent advances in reasoning models have demonstrated remarkable capabilities on mathematical and coding tasks, but their effectiveness in embodied domains remains largely unexplored. |
| Approach: | They propose a reasoning model for interactive embodied tasks that synthesizes 9.3k coherent Observation-Thought-Action trajectories containing 64k ego-centric images and 90k diverse reasoning processes. |
| Outcome: | The proposed model outperforms existing visual reasoning models by +9%, 24%, and +13% on long-horizon tasks. |
Lightweight and Faithful Visual Condition Checking in Behavior Trees via Expert-Regularized Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing behavior trees are not suitable for high-dimensional perceptual inputs such as images or language. |
| Approach: | They propose a framework that leverages expert-regularized reinforcement learning to preserve semantic faithfulness while employing a factorized policy that aggregates sequential condition-node decisions into a single decision unit. |
| Outcome: | The proposed framework outperforms imitation learning and reinforcement learning but risks misalignment of condition nodes with intended semantics and poor credit assignment. |
Doc-V*: Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA (2026.acl-long)
Copied to clipboard
Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai
| Challenge: | Existing OCR-free approaches to document visual question answering are brittle and passive. |
| Approach: | They propose an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation. |
| Outcome: | The proposed framework outperforms open-source and proprietary models in five benchmarks and improves out-of-domain performance by 47.9% over baseline. |